According to the background art, it is known to acquire different views of a same scene in a single snapshot with a plenoptic camera, also called light-field camera. A direct application of such a plenoptic camera is 3D reconstruction. Indeed, after demultiplexing of the raw image acquired with the photosensor array of the plenoptic camera, the recovered views of the scene are already in epipolar geometry horizontally and vertically, so the disparity between them can be estimated without stereo rectification. This is a huge advantage compared to binocular 3D stereo reconstruction from images captured with a conventional camera.
Nevertheless, estimating disparity from views resulting from the demultiplexing of the raw image suffers from several issues. For example, the views resulting from the demultiplexing offers only one single color information for some of the pixels of the views while other pixels do not have any color information associated with them: the spatial color sampling of such views is often jagged and incomplete, which leads to erroneous disparity estimation. Also the demultiplexing of the raw image often rely on interpolation (de-multiplexed pixel with non-integer coordinate), which leads to erroneous disparity. A solution to such disparity estimation issues is to first demosaice the raw image before demultiplexing it in order to have full color information for each pixel of each view of the scene resulting from the demultiplexing. But performing the demosaicing before the demultiplexing may lead to other issues, such as inter-view crosstalk. Indeed, as to recover the full color information for one given pixel of the raw image, pixels belonging to the neighborhood of this given pixel may be used, even if these neighboring pixels belongs to other view(s) than the view of the given pixel. Estimating the disparity on such views suffering from inter-view crosstalk may also lead to disparity errors.
Moreover, determining the focus associated with object(s) of the acquired scene may be a heavy process subject to precision issues.